DEPARTMENT  OF  DEFENSE 


MANUFACTURING  AND  INDUSTRIAL  BASE  POLICY  (MIBP) 


Identifying  and  Mitigating  Industrial  Base 
Risk  for  the  DoD:  Results  of  a  Pilot  Study 

Sector-by-Sector,  Tier-by-Tier  (S2T2) 
Fragility  and  Criticality  Assessments 


Sally  Sleeper,  Ph.D.1 
John  Starns,  D.Sc.  2 
Eugene  Warner,  Ph 


AFCEA  Acquisition 
Research  Symposium 

May  14-15,  2014 


&L),  Manufacturing  and  Industrial  Base  Policy 
rthrop  Grumman  Technical  Services 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

MAY  2014  2' REPORT  TYPE 

3.  DATES  COVERED 

00-00-2014  to  00-00-2014 

4.  TITLE  AND  SUBTITLE 

Identifying  and  Mitigating  Industrial  Base  Risk  for  the  DoD:  Results  of  a 
Pilot  Study:  Sector-by-Sector,  Tier-by-Tier  (S2T2)  Fragility  and 

Criticality  Assessments 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
OUSD(AT&L)  Manufacturing  and  Industrial  Base 

Policy, Washington, DC, 20301 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

ARSTRATT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  3S 

unclassified  unclassified  unclassified  Report  (SAR) 

23 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Outline 


Scoping  the  Problem 

★  Methodology 

★  Results 

★  Findings 
Next  Steps 


Acquisition,  Technology  and  Logistics 
Manufacturing  and  Industrial  Base  Policy 


MIBP  Mission:  Ensure  access  to  robust,  secure  and 
innovative  industrial  capabilities  to  fulfill  short-  and  long-term 

National  Security  requirements 


Constant  FY2014  Dollars  in  Billions 


Budget  Swings  Have  Significant 
Consequences  for  the  Industrial  Base 


DoD  Investment  Outlays 
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Contractors  &  their  vendors 

during 

Upswings: 

•  Acquire  resources  to 
address  their  schedule  and 
performance  requirements 

•  Resources  may  be  limited 
due  to  demand 

Downswings: 

•  Decide  how  much  of  that 
capability  they  can  afford  to 
maintain  or 

•  Decide  to  exit  the  defense 
market 


Will  Warfighter  Get  Support  When 
Needed? 

Capitalism:  Markets  will  right-size  based  on  demand 

*  Companies  enter  when  it  is  profitable,  and  exit  otherwise 

Many  capabilities  used  by  defense  exist  during 
upswings  and  downswings 

*  Capabilities  “easy”  to  reproduce;  low  barriers  to  entry 

*  Market  has  alternatives  or  substitutes 

But  some  capabilities  are  sensitive  to  defense 
procurement  swings 

*  Small  or  no  market  without  defense 

*  Little  slack  available  during  upswings 

*  Difficult  to  balance  capital  investments,  specialized  labor  with 
large  budget  changes 


S2T2 
Assessment 


Approach  for  IB 


S2T2  Program  Vision 


Develop  a  collaborative,  repeatable, 
fact-based  DoD-wide  internal  ability 
to  evaluate  the  impact  of  acquisition 
decisions  on  the  industrial  base  (IB) 

•  Monitor  and  assess 

•  industry  readiness,  competitiveness, 
ability  to  innovate,  and  financial  stability 

•  Supply  analysis  to  decisionmakers 

•  to  support  investment  decisions  for 
preservation  and  transformation  of  the 
IB  to  support  national  security  objectives 


S2T2  Program  Objectives 

Integrate  IB  considerations  into  acquisition 
strategy  decision  making 

Identify  successful  IB  management  efforts 

Reduce  duplication  of  effort  in  OSD  and 
Services 

Establish  early  warning  indicators 

Identify  Industrial  Base  risk,  particularly  at 
the  lower  tiers  of  the  supply  chain 


Leverage  a  statistically-validated  &  standardized  Fragility  &  Criticality  (FaC) 
assessment  process  to  analyze  risk  across  the  tiers  of  the  Industrial  Base 


S2T2  FaC  Process 


Process  Activity 


Action 


Outcome 


Select 

Sector/SubSector 

l 

Search 

Available  Data 


Scope  the  problem  (existing  Preliminary  Sector  Taxonomy 

risk  assessments;  program 

shutdowns) 

Identify  IB-related  risks  &  Expanded  Taxonomy  and 

related  capabilities/products  Product  Supplier  Pairs 
Identify  suppliers  and  market 


FaC  Screening/Filtering 

I 


Focused  set  of  IB-related  risks  Screened  IB/Issues 
for  further  assessment  Capability-Supplier  Pairs 


Conduct 

FaC  Matrix  Assessment 


Facilitated  scoring,  based  on 
standardized  criteria,  by  SMEs 


SME  “deep  dive”  into  IB  risk 
areas;  facility  visits 


High  Risk  IB  Issues 


S2T2:  iterative,  repeatable,  collaborative,  fact-based 


FaC  Pilots  Provide  Insights  and  Direction 

★  Key  lessons  learned 
Test  analytic  framework 

★  Refine  factor  definitions 


FaC  Pilots:  Some  Key  Lessons  Learned 

FaC  process  well-suited  to  assessing  a  portfolio  of  suppliers 
supporting  similar  capabilities,  and  deriving  cross-cutting 
solutions 

*  Program-specific  FaC’s  that  repeat  IB  assessments  reveal  little  new 
information 

Some  criticality  factor  definitions  ambiguous,  redundant;  Some 
fragility  factors  are  difficult  to  obtain 

Insure  the  taxonomy  is  standardized 

*  Map  results  to  taxonomy 


FaC  Pilots:  Test  Analytic  Framework  <iof2) 

Conclusion:  Pilots  improved  understanding  of  factor 
definitions  and  scoring 

Combine  FaC-Matrix  data  from  pilot  assessments 

*  Consistent  scoring? 

*  Redundant  factors? 

*  Contribution  to  criticality/fragility  constructs? 

*  Missing  factors? 

Empirical  tests:  Weights 

*  Different  weights  employed,  but  same  core  set  of  factors 
deemed  “most  important” 

*  Applying  uniform  weights  to  combined  data  set  do  not  alter 
the  observed  outcomes  from  individual  FaC’s 


FaC  Pilots:  Test  Analytic  Framework  <2of2) 

Empirical  tests:  Redundant  factors 

*  Pilots  indicated  redundant,  difficult  factors 

*  Eliminating  sub-set  of  fragility  factors  from  combined  data  set 
do  not  alter  the  observed  outcomes  from  individual  FaC’s 

Statistical  tests:  Criticality 

*  Factor  analysis:  Identified  2  unique  sets  of  factors  consisting 
of  5  items:  Defense  unique,  skilled  labor,  design  intensity, 
reconstitution  cost,  availability  of  alternatives 

*  Cronbach’s  Alpha:  5  items  reliably  measure  the  same  latent 
construct 

*  Combined  construct:  “niche  capabilities,”  “difficult  to  replace 
if  lost” 

*  Empirical  application  of  sub-set  of  criticality  factors  do  not 
alter  the  observed  outcomes  from  individual  FaC’s 


FaC  Pilots:  Refine  Definitions,  Weights 

Criticality:  “critical  niche  products,”  “difficult  to 
replace  if  lost” 

*  Pilots  suggest  missing  factor:  equipment  and  facility 

*  Pilots  suggest  clarification:  “Reconstitution”  to  consider 
impact  on  DoD  relative  to  time  to  restore  the  capability,  if  lost 

*  6  factors  consistent  with  construct,  equal  weights 

★  Fragility:  “risk  of  exit  by  current  supplier,  risk  current 
market  cannot  meet  requirements” 

*  Pilots  suggest  doing  deep  dive  to  gain  factory-floor 
perspective  when  warranted 

*  4  factors  consistent  with  construct,  equal  weights:  2  supplier, 

2  market 

*  Improve  data  collection  for  supplier  information 


S2T2  Fragility  and  Criticality  Criteria:  Refined 
based  on^Y13  Pilot  Assessments 

Capability  =  technology,  part,  component,  product 


Criticality: 

•  Characteristics  that  make 
a  specific  Capability 
difficult  to  replace  if 
disrupted 

Defense  unique  capability 
Skilled  labor  requirements 
Defense  Design  requirements 
Facility  &  Equipment  requirements 
Reconstitution  time 
Availability  of  Alternatives 


Fragility: 

•  Characteristics  that  make  a 
specific  Capability  likely  to 
be  disrupted 

Financial  Outlook  (Current  provider) 
DoD  Sales  (Current  provider) 

Firms  in  Sector  (Existing  market) 
Foreign  Dependency  (Existing  market) 


S2T2:  collaborative,  iterative,  repeatable,  fact-based 
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Tools  for  FY14  FaC  Assessments 

*  FaC-List  -  collect  information  so  that  it  is 
more  easily  shared 

★  FaC-Matrix  -  include  options  to  identify  and 
isolate  areas  of  interest 

★  FaC-Validation  -  template  to  document  results 
more  consistently 

*  FaC-Summary  -  guidelines  to  communicate 
findings 


Next  Steps  for  FaC  Process 

Conduct  pilot  FaC  assessments  for  skills 
*  Improve  capture  and  sharing  of  FaC  data 
Data  mining  to  improve  fragility  ratings 


Leadership  Awareness 


Deputies  Management  Action  Group  (DMAG) 
chaired  by  DEPSECDEF 

*  Spring  2013  -  Include  Industrial  Base 
Considerations  in  POM  planning. 

*  Late  2013  -  Resource  Decision  Memorandum  for 
remedial  actions  on  imminent  industrial  base  risks. 

*  Late  2014  -  Industrial  Base  DMAG  requested  by 
USD  (AT&L),  to  be  informed  by  2014  assessments. 


2014  Industrial  Base  Assessment  milestone:  DEPSECDEF  DMAG 
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Four  Principle  Types  of  Decision  Points 


Individual  Program  Acquisitions 

*  Milestone  A 
4  Milestone  B 
4  Milestone  C 

*  Termination 

Budget  Cycle  /  Portfolio  Reviews 

4  Annual/Bi-annual 
4  Secular  defense  build/shrink 
4  Annual  Report 

Long-Term  Working  Groups 

*  Defense  Production  Act  (DPA)  Study  Groups 
4  Supply  Chain  Risk  Management 
4  DIB  Information  Assurance 
4  Critical  Infrastructure  Program 
4  Space  Industrial  Base  Council/Critical  Technologies  Working  Group 
4  Joint  Industrial  Base  Working  Group  (JIBWG) 

4  NATIBO 

Emergent  Issues 

4  CFIUS/M&A 

4  Individual  Company  Issues 
4  Individual  Program  Issues 
4  External  event 

*  Surge 


Having  current  &  complete 
analyses  of  the  IB  enhances 
DoD  Senior  Leader  decision¬ 
making  &  allows  timely 
identification  of  the  impacts  of 
program  changes! 


Assumptions 

DoD  will  never  have  complete  Industrial  Base  visibility 

*  But,  most  areas  of  the  industrial  base  are  not  critical  or  fragile. 

*  S2T2/FaC  approach  quickly  winnows  out  non-critical  capabilities  to  focus 
attention  -  and  resources  -  on  areas  of  potential  risk 

IB  Assessment  Process 

*  Should  take  maximum  advantage  of  information  routinely  produced  as 
part  of  the  normal  business  process 

*  Must  accommodate  flexibility  where  warranted  and  uniformity  where 
required 

*  Is  iterative  and  seeks  to  continuously  expand  DoD’s  insight  into  IB 
capability  and  constraints. 
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Rating 


iTechnology,  part.  Criticality  =  Characteristics  that  make  a  specific  product  or  service  difficult  to  replace  if  disrupted. 


Skilled  labor 
requirement 


Facility  & 
Equipment 
requirements 


Reconstitution 

Time 


Availability  of 
Alternatives 


Q:  To  what  degree  is  the  market  for  this  capability  commercial? 


Defense  Unique 


80%  or  more 
commercial  market 


30-60%  commercial. 
Low  barriers  to  enter 
market 


Minimal  special  skills. 
Expertise  commonly 
available  or  easily 
obtained. 


Specialized  skills,  but 
processes  well 
documented.  No 
workforce  issues. 


30-60%  commercial. 
Significant  and  costly 
barriers  to  entry 


20%  or  less  commercial. 
Relatively  low  barriers  to 
entry 


20%  or  less 

commercial.  Significant 
and  costly  barriers  to 
enter  market 


Q:  To  what  degree  are  specialized  skills  needed  and  available  to  integrate,  manufacture  or  maintain  this  capability? 


Highly  specialized  skills, 
no  workforce  issues  near 
term. 


Highly  specialized  skills, 
potential  workforce  issues 
near  term  (e.g.  limited 
specialists  available) 


Highly  specialized  skills 
and  workforce  issues 
anticipated  (e.g. 
Limited  specialists; 
diminishing  workforce) 


Q:  To  what  degree  is  defense-specific  knowledge  required  to  reproduce  this  capability,  an  alternative,  or  the  next 
generation  design? 


Defense  design 
requirements 


Designs  are 
commercially  available. 
Minimal  defense-related 
knowledge  required. 


Designs  are 
commercially  available, 
but  some  defense- 
specific  (non¬ 
commercial)  knowledge 
required. 


Specialized  and  defense- 
specific,  no  workforce 
issues  near  term 


Specialized  and  defense 
specific,  potential 
workforce  issues  near  term 
(e.g.,  limited  availability) 


Highly  specialized  and 
limited  workforce  (e.g., 
unique  defense 
parameters,  security 
clearance,  proprietary 
practices) 


Q:  Are  specialized  equipment  or  facilities  needed  to  integrate,  manufacture,  or  maintain  this  capability? 


Minimal. 
Equipment/facilities  are 
common 


Limited.  Alternative 
sources  can  produce 
similar  products. 


Moderate,  (e.g., 
qualification  of  production 
line;  specialized  skills  or 
technology) 


Specialized 


Q:  What  is  the  impact  on  the  DoD  in  time  to  restore  this  capability  if  it  is  lost? 


Minimal  time  impact  to 
restore 


Limited  time  impact  to 
restore 


Moderate  time  impact  to 
restore 


Significant  time  impact  to 
restore 


Highly  specialized 
equipment/facilities  are 
required 


Severe  time  impact  to 
restore. 


Q:  To  what  degree  are  cost,  time,  and  performance-effective  alternatives  available  to  meet  DoD  needs? 


"Drop-ins"  exist  and  are 
currently  used  in  other 
programs 


Alternatives  exist. 
Low/limited  impact  to 
substitute 


Moderate  impact  to 
incorporate  substitute 
alternatives 


Significant  impact  to  use 
substitute  alternatives 


Severe  impact:  Limited 
or  no  reasonable 
alternatives  or 
workarounds  exist 


Product  Fragility  Ratings  - 

v5:  Rev  Date  01Nov2013 

Rating 

1 

a  |  b 

c 

d 

e 

Fragility  =  indicator  of  whether  the  Department  will  receive  what  it  needs  when  it  needs  it  from  (1)  the  current  provider,  (2)  the 

existing  market 

Financial  Outlook 
(current  provider) 

Q:  What  is  the  risk  of  this  facility  going  out  of  business  or  exiting  the  market  for  this  capability? 

Very  low  risk.  Viable 
and  stable,  (e.g. 
excellent  overall 
financial  rating  and 
strong  product  line) 

Some  risk. 

Moderate  risk.  (e.g. 
financial  indicators  risk 
or  risk  of  the  facility 
ceasing  capability 
production  are 
moderate) 

Strong  risk. 

Severe  risk. 
Imminent  exit  (e.g., 
firm  going  out  of 
business  or  facility 
leaving  the  business 
line) 

DOD  Sales 
(current  provider) 

Q:  How  much  total  sales  for  this  facility  are  from  DoD  contracts? 

Mixed  DoD  and  non- 
DoD  Market 

Significant  but  not 
dominant  DoD  ornon- 
DoD  market 

Dominance:  >80%  or 
<20%  in  total  DoD 
sales 

Firms  in  Sector 
(existing  market) 

Q:  How  many  firms  currently  participate  in  this  firm's  market  for  this  capability? 

More  than  10 

6  to  10 

3  to  5 

2 

1 

Foreign 
Dependency 
(existing  market) 

Q:  What  is  the  dependence  on  foreign  sources  for  this  capability? 

Domestic  suppliers 

1  or  2  domestic 
supplier(s),  foreign 
source(s)  may  exist 

Current  foreign  source, 
but  domestic  supplier(s) 
exist 

Only  foreign 
source(s)  exist, 
potential  for  domestic 
source 

Only  foreign 
source(s)  exist 

Statistical  Testing  Results 


★  Criticality  Items 

*  Factor  Analysis  identified  3 
factors  (eigenvalue^) 

•  Note  Factor  Analysis  “factor” 
interrelated  variables 


Factor  Matrix 

Factor 

1  2 

3 

FAC01 

.648 

-.465 

.108 

FAC02 

-.178 

.396 

.112 

FAC03 

.655 

-.196 

.004 

FAC04 

.734 

.002 

.013 

FAC05 

.497 

.514 

.411 

FAC06 

.296 

-.077 

.144 

FAC07 

.032 

.140 

.119 

FAC08 

.655 

.259 

-.229 

FAC09 

.314 

.292 

-.517 

*  Factor  1:  Defense  Unique,  Skilled  Labor,  Design 
Intensity,  Reconstitution  Cost 


*  Factor  3:  Long-lead  time  (inverse) 
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